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TOWARDS A COMPLETE THEORY OF INFORMATION. 
BEYOND 
THE SHANNON’S COMMUNICATION THEORY 


Dan Alexandru IORDACHE!” 


Beyond the statistical (and numerical) aspects of information, there are several other 
important aspects that are not at all examined by the Shannon’s theory. Starting from the 
proposals of Werner Gitt, this work examines a 6 levels Information concept, which 
involves also a 4" (in the Complexity order) level intended to the quality check of the 
transmitted information (by means of the identification and elimination of all ambiguities 
and possible misunderstanding). Following a very short examination of the basic notions 
of the Shannon’s statistical information theory [of the definition of the Shannon's 
information amount (entropy), inclusively], the complex physical process of the 
continuous (Bremsstrahlung) X- radiation was chosen to study and exemplify the effective 
content of the successive steps of the Information concept. 


Key words: Shannon’s Information Theory, Syntax, Semantics, Information Accuracy, 
Pragmatics, Apobetics 


1. Introduction 


Given being that the applications of Shannon’s information theory [1]-[5] are well 
known, we will examine now mainly the criticisms of this theory and the 
possibilities to achieve some suitable completions. 


So, the German cyberneticist Bernhard Hassenstein criticized the Shannon’s 
information theory in the following words: “It would have been better to devise an 
artificial term, rather than taking a common word and giving it a completely new 
meaning” [6]. 


According to the information specialist Werner Gitt “no science, apart from the 
communication technology, should limit itself to just the Claude Shannon’s 
statistical level of information” [7]. 


Several authors repeatedly pointed out that Shannon’s definition of information 
encompasses only a very minor aspect of information [8], p. 50. Even Warren 
Weaver — the Mathematics professor and main collaborator of Claude Shannon 
has written [3]: “Two messages, one of which is heavily loaded with meaning and 
the other which is pure nonsense, can be exactly equivalent ... as regards 
information”. 
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Similarly, the German information scientist Karl Steinbuch has found [9]: “The 
classical (Shannon’s) theory of information can be compared to the statement that 
one-kilogram of gold has the same value as one kilogram of sand”. 


We have to mention also that many authors erroneously elevate Shannon’s 
information theory to the syntactic level, as in the frame of Ernst von 
Weiszacker’s finding [10]: “The reason for the ‘uselessness’ of Shannon’s theory 
in the different sciences is frankly that no science can limit itself to its syntactic 
level”. According to the German biologist G. Osche [11], the Shannon’s theory is 
unsuitable from a biological point of view: “In cybernetics, the general 
information concept quantitatively expresses the information content of a given 
set of symbols by employing the probability distribution of all possible 
permutations of the symbols. But the information content of biological systems 
(genetic information) is concerned with its value and its functional meaning and 
thus with the semantic aspect of information, with its quality”. 


2. Structure of the Information Concept 


2.1. 


Accepting the opinions of the work [7] (page 60, figure 12) and completing them 
with a 6"' Information level, intended to the check of the Information accuracy 
(identification and elimination of ambiguities and possible misunderstanding), we 
propose the structural scheme of the Information concept reported below by 


Figure 1. 
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Fig. 1. The 6 Information Levels (besides the 5 Information levels indicated by work [7], as this 
paper considers as compulsory level that intended to the check of the Information accuracy - the 
4" Jevel in this scheme; there are ensured so good results of the received information applications). 
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3. The basic notions of the statistical (Shannon’s) information level 


Consider the statistical set 01,0 ,...0,, of the possible discrete states (or processes) 
of a certain studied physical system, and let pj, p,...p, be the probabilities of 
achievement of each of these states (processes). Of course, if this statistical set is 


n 
complete, then: )p;=1. The usual (Shannon-Khinchin [1]-[5]) definition of the 
i=l 
nondetermination (uncertainty) degree H(p), p2,...pn) associated to this statistical 
set is given by the requirements of: 
a) symmetry: H(p, p2.---Pn)=H(p2; Pis--Pn), ete. (3.1) 
b) obtainment of the maximum value of the  non-determination 
(uncertainty) degree for: 
1 
a Pee om (3.2) 
c) invariance of the uncertainty degree at the addition of an impossible event 
(Pn=0): Hp, P2.--Pn%) = APL, P2>--Pn)> (3.3) 
d) continuity of the function H(p, p2,.--Py) relative to the values of the 
probabilities p,, p2,...Pn> 


e) linearity of the non-determination (uncertainty) degree, expressed by 
means of the relation: 


H(C,C)=H(O)+ Sp; H(Clq), (3.4) 


i=l 


where H(C, C’) is the uncertainty degree corresponding to the Cartesian product 
of the sets C, C’ of the states o; and o' p> correspondingly, while H(C'|o;) is the 


uncertainty degree corresponding to the set C’, in the conditions of the state o; 
achievement. 


One derives rather easy (see e.g. [12], pp. 210-212) that the uncertainty function 
fulfilling the above indicated conditions (3.1) — (3.4) is given by the expression: 


n 
A (Pj, P2,---Pn) = —4- LP; logy Dj , (3.5) 
i=l 
where a and b are almost arbitrary constants, that satisfy the conditions: a > 0 and b > 1. 


From the expression (3.5), one finds that the uncertainty function H(pj, p2,...py) 18 
the theoretical average of the so-called information entropy, defined by the relation: 


S; =-a- logy p; . (3.6) 
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Similarly, for a continuous distribution, the information entropy will be given by 
the expression: 


S =—a-logp, + constant, (3.7) 


where go is the probability density corresponding to the considered continuous 
distribution. 


The basic information entropy (amount) unit — the famous bit (from the complete 

denomination binary information unit) corresponds to the values a = | and b = 2. 

Given being that in December 1900, the illustrious German scientist Ludwig 

Boltzmann has found that the value a=kp aoa unites the Claustius’ 
A 

thermodynamic entropy (see e.g. [15], p. 276) with the natural (b = e = Euler’s 

number) logarithm of the probability density of the physical state: 


S=—kp-logp gw + constant, (3.8) 
one finds that both for the: 
a) quantity name: entropy, as well as for: b) its statistical expression, 
the statistical Physics has a priority of almost 50 years relative to the Shannon’s 


Statistical information theory. 


4. Examples concerning the Information Concept for the Particular Case of 
the Continuous (bremsstrahlung) X-radiation 


4.1. The Syntactic level 


According to [8], p. 61 “Syntax is meant to include all structural properties of the 
process setting up information”. 


a) Experimental Part 
In order to leave the exclusively statistical (quantitative) description of the 


information by means of Shannon’s theory, it is useful to consider the 
experimental part of any usual information source, of the Continuous 
(bremsstrahlung) X-radiation, particularly (see figures 4.1, 4.2 and 4.3; see also 
[24], pp. 92-95). 


Glass Vacuum Envelope 


Useful’X-ray beam 
Fig. 4.1. A simplified Schema of a typical X-ray vacuum tube. 


Towards a Complete Theory of Information. 
Beyond the Shannon’s Communication Theory 23 


The continuous spectrum is only about 10” of the inten- 
sity at the center of the line 
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Fig. 4.2. Low-resolution X-ray spectrum of Fig. 4.3. One of the Mo X-rays lines 
molybdenum. [23] observed with a high-resolution spectrometer. 


b) The Physical Dimension — specific feature of the Syntax of the 
description of complex systems in nature sciences 


It is well known that the Syntaxes of any language (modern [16] or antique [17]) 
distinguish the subject (examined system), the verb (action), the adverb (the 
conditions in frame of which the considered systems operate), etc. This 
corresponds in Physics to distinguish clearly the cause (e.g. the force), the effect 
(e.g. the acceleration), the system inertia parameter (e.g. its mass), etc. 


In practice, the trend — for the extremely Complex systems — to simplify the 
expressions of the corresponding relations could lead sometimes to considerable 
difficulties. As an example, the elaboration [18] by the well-known British 
specialist in Mathematical Physics — Douglas Hartree (1897-1958) of an 
dimensionless (atomic) system of physical units contributed both to the: 


a) elaboration of certain programming languages (around 1950, as the 
Fortran one, its compiler being already in use in 1957), which achieved (as the 
Fortran language) the descriptions of physical relations by means of some 
numerical computing instructions, and to the: 


b) formulation of some Physics works (studying certain processes of high 
complexity, as the continuous X-ray spectra) by specific highly simplified (in the 
frame of Hartree’s dimensionless system) relations, which unfortunately didn’t 
allow to distinguish the nature of the described physical quantities. 


For example, all quantitative expressions of the classical works concerning the 
continuous (Bremsstrahlung) X-radiation [19], [20] are dimensionless, hence they 
are out of Syntax, given being in nature sciences the Syntax requires specific 
physical dimensions [Q] for any quantity Q (e.g. [force] = M-L-T?, 
[acceleration] = L-T~’, [speed] = L-T~!, etc, where M, L, T., ...are the physical 
dimensions of the mass, length, time, etc.), to distinguish easily and quickly 
among the terms of any physical expression. 
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For this reason, the appendices Al and A2 of the work [21] was intended 
(pp. 78-81) to the translation of all dimensionless expressions of the classical 
works dedicated to the continuous (Bremsstrahlung) X-rays in an units systems 
with physical dimensions, namely in the classical system CGS. 


4.2.The Semantics level 


According to [8], p. 71, the Semantic level refers to the understanding of the exact 
meaning of the different sentences, or of the relations from the Nature Sciences. 


The understanding of the complex physical processes begins from their 
elementary (microscopic) level. 


So, for the continuous (bremsstrahlung) X-radiation, the corresponding 
microscopic process consists in the collision of an electron of kinetic energy E, 
and linear momentum p, with an atomic ion (or nucleus), emitting a photon of 
linear momentum & (energy hv =k-c) and polarization direction j (perpendicular 
on k ), afterwards the electron emerging with the linear momentum jp and the 
energy E. 

The basic tools for the understanding of this elementary (microscopic) process 


consist in the corresponding Feynman-Dyson diagrams (see fig. 4.4) and the 
associated vector diagrams (see e.g. fig. 4.5). 


Fig. 4.4. Fig. 4.5. 


The left side diagram corresponds to a previous interaction of the incoming 
electron with the atomic field, followed by the emission of a photon, while for the 
right side diagram the photon emission precedes the electron interaction with the 
atomic field. 


The parameters f,, f> correspond to the 4 dimensional linear moments associated 
to the internal electronic line of the first and of the second diagram, respectively, 
7=Po-p-—k is the linear momentum given to the nucleus as a result of the 
process of the incoming electrons braking, while -g is the linear momentum 
“absorbed” from the atomic potential. 
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Taking into account the huge number of sub-fields corresponding to the Complex 
systems evolutions, the thorough (with the examination of all particular 
conditions) derivation of their main relations is absolutely necessary. That is why 
the work [21] dedicated (from a total of 15 appendices) not less than 13 ones (the 
other 2 being dedicated to some Syntax matters) to the examination of the basic 
relations used in the frame of the continuous (bremsstrahlung) X-rays quantum 
theory: 


Appendix 3: Derivation of the polarization degree P of X-rays, 
Appendix 4: Calculus of the limits of P for p/p, >0 and p/p, >1, 
Appendix 5: Derivation of the expression 
q° sin? LE po sin” Bo +p? sin? G-2p-posinYsinY, cosy, Where: G=p,-p and 
3, 9, are the angles between the linear moments k, p and k, p,, respectively, 


while g is the angle between the planes k, p and k, py, 


Appendix 6: Derivation of the expression of the angular distribution of the 
continuous (Bremsstrahlung) X-radiation, 


Appendix 7: Derivation of the expression of the angular distribution of the 
diffused electrons, 


Appendix 8: Derivation of the expression of the integrated cross-section, 
neglecting the screening, 


Appendix 9: Derivation of the expression of the integrated cross-section, 
taking into account the screening, 


Appendix 10: Calculus of the average energy loss due to the continuous 
(bremsstrahlung) X-radiation, 


Appendix 11: Retardation effect on the angular distribution of the 
continuous (bremsstrahlung) X-radiation, 


Appendix 12: Calculus of the expression of q° (where 7 = Po-P)s 


Appendix 13: Calculus of the expression of g* in terms of 
E= LP, Po —k)), 


Appendix 14: Calculus of the Bethe-Heitler [19], [20] parameter I in the 
case of w<< p< py, 


Appendix 15: Calculus — in the case w< p<<p, - of the differential 
cross-section o(v,é,.9,)-dv-dé-ddI,. 
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4.3.The Accuracy level 


The matter referring to the possibilities to improve the accuracy of the 
information concerning certain states or processes of some complex systems is 
examined in our work [25]. For this reason, we will focus here on the possibilities 
to identify and avoid the (i) ambiguities (see also [26], [27]), (ii) misinformation. 


a) No ambiguities means: Accurately known probability densities of 

the individual values of the studied parameters 
While the value of the unknown of a mathematical problem rightly formulated is 
obtained exactly by means of this problem solution, the most probable individual 


value (named also “the true value”, or the “mathematical hope”) ‘p of a certain 
physical parameter p cannot be never obtained exactly! 


In steady states, the function of the probability density has to be known accurately (see 
fig. 4.6), even if the individual values of the studied parameters have random values. 


b) Typical elementary object 
While in Mathematics the typical elementary object (the problem unknown) is a 
number or an uniquely defined segment, in the Nature Sciences this elementary 
object is a physical parameter yz, described by a certain probability density P(p) 
of the individual values (see fig. 4.7). 


For this reason, the definition of the true information amount would correspond to 
the superposition area of the functions of the probability density normalized to 1, 
and corresponding to the measurements of the studied physical parameter, 
respectively. 
Amount of true information = Distributions of the 
3 I —— . individual values for: 
, Eube wa Enter Cae experimental results 
distribution is normalized to 1 DD or ths tranenaitied 
j | \ information 
- true physical parameter 
sy 


Amount of True Information < 0 


represents MisInformation! 
P | 


Fig. 4.6. Function of the probability Fig. 4.7. Definition of the true amount 
density of the individual values of information obtained by means of 
of a physical parameter. measurements. 


4.4.The Pragmatics level 


Between the 2 types of X-rays spectra (continuous and of lines; see fig. 4.2), the 
continuous (Bremsstrahlung) type is that suitable for the evaluation of the periods of 
a crystalline lattice. In this aim, there are necessary the basic theoretical relations, as 
well as some procedures of selection of the convenient X-radiation wavelength. 
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a) Derivation of Laue’s equations and of the Bragg and Bragg law 


It is well-known that the diffraction maxims corresponding to a_ parallel 


electromagnetic beam of wavelength A, incident along the unit vector 1; upon an 
1D-diffraction grating of vector period d , have the directions 1, so that: 

d(cosa@ —cosaq ) =d(\g —1;) =m, 
where mm is the diffraction order (a positive, or negative integer). 
Consider now the diffraction of a parallel electromagnetic beam (of wavelength 7 ), 
incident along the direction 1; upon a 3D-lattice (a crystalline lattice, particularly). 
Let be a,b and @ the (non-coplanar) basic vectors (periods) of the considered 
3D-lattice; the intensity of the diffracted beam along the direction ly will be 
maximum if (concomitantly): 


a(lg—-])=m'A, bUg-lj)=n's, Edg-])=p'A, (4.5.1) 


where m', n' and p' are integer numbers, called the Laue numbers (the above 
conditions are called the Laue's equations). Let be D = I.d.(m', n’, p') the largest 


min=— and p= ay Starting from the 
D D 


divider of integers m', n', p' and rege 


Laue's equations, one finds that: 


Egle ise 1)= ea 1)=DA , (4.5.2) 
m n P 
and: CO Ge se Sas psesSaGens0. 14653) 
mon n p pom 


Miller indices m, n and p) defined by the vectors ab be and ee 
mn np pom 


build up starting from the same lattice node O (see Fig. 4.8). 
c 


Crossing with Ke ‘ 
\ 


the Miller plane 


a 


Fig. 4.9. 
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Suppose now that the sheet coincides with the plane of the unit vectors 1;,1) , and 
that the horizontal is the crossing of the plane of Miller indices m, n, p with that of 
the vectors 1; and 1, (fig. 7.73), the unit vector of the normal on the Miller plane 
(m, n, p) being denoted by 1,,. Because: (1y — 1;) x 1, =0, one obtains: 


sin 6; =|] x, | =(Iq x Ih =sin@,, and: Ji ~7j[=2sinS, (4.5.4) 


where 6 is the deviation angle of the diffracted beam relative to the incident one. 
From relations (4.5.2) and (4.5.4b), one obtains: 
ee, 6 ro 
~ Ti fla — i] =2dmnp sin 


5e SDRadd Gian ped, (4.5.5) 


m 
where dimyp is the inter-planar distance between the Miller planes (m, n, p). The 


relation (4.5.5) is named the Bragg and Bragg law (of the diffraction on 
3D-lattices). 


b) The reciprocal lattice 


The calculations of the inter-planar distance d of the change of the wave- 


mnp ? 
vector: Ak = ay —1;) due to the diffraction on a 3D-lattice and others, can be 


simplified using the notion of reciprocal lattice. 


p= 


Let be: a'= = 6x2), " (¢xa), z= = @xb), (4.5.6) 


the basic vectors of the reciprocal lattice, V =a-(b x@)=b -(€xa@) =e -(a@xb) 
being the volume of the unit cell, and: 


p'= m'a't+n'b'+ p'e'= Dima'tnb'+pe') = D-p (4.5.7) 


- an arbitrary position vector inside the reciprocal lattice. 


One finds that: g ‘p= Z p= z -p=2zn, (4.5.8) 
m n Dp 
ae ag me ae FO 
therefore: ( )p = ( )p =( yeo=03 (4.5.9) 
min n p pom 


it results that the position vector p is parallel to 1j — 1; and perpendicular also on 
the Miller plane (m, n, p). 


As an application, the inter-planar distance dj,,) can be expressed as: 


dmnp = - mL (4.5.10) 
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Similarly, the change of the wave-vector due to the diffraction on the 3D-lattice 


. 2n 2n 2n 0 2x- DA = Lape 
1S: Ak = ly -1)= Ly - |= 1, -2si = 1: =Dpl|, =Dp= p'. 
q (la i) 7A n|la Hl qn ay qn dg Ply P=P 
2 2 
Finally, from the relation: a =k’? = (k + p) = a + 2k pitp ; 
A A 
one finds that: po =2k pep", (4.5.11) 


which is the equivalent form (for the reciprocal lattice space) of the Bragg and 
Bragg law. 


a) The Ewald’s geometrical construction 


Taking into account that only 2 from the 3 directory cosines: 
a=lg-l,, B=lg-ly and: y=Q-l, (4.5.12) 


of the 1, direction are independent (because: a? + Bp? + y? =1), it results that - 
for arbitrary 1, and 4 - the Laue's equations (4.5.1) cannot be fulfilled 
concomitantly. In order to find the wavelength 4 which can accomplish 
concomitantly these equations for given 1; and p', the following (Ewald) 
construction can be used. Consider the space of the reciprocal lattice and the 
vector p' inside this space (fig. 4.10). 


Fig. 4.10. 


Let J be the starting point of p' and O - the crossing of the mediator plane of p' 


and of the straight-line parallel to 1;, passing through J. One finds easily that 
k =OI satisfies the Bragg law (4.5.11). It results that: 
gees (4.5.13) 

k Ol 


4.5.The Apobetics level 


Given being any piece of information has a purpose, the last and highest level of 
information refers to this (teleological) aspect: the question of purpose. 
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The term “apobetics” was introduced in 1981 by W. Gitt [28], starting from the 
Greek word apobeinon = result, success, conclusion. 


As it concerns the particular example chosen by us: that of the continuous 
(Bremsstrahlung) X-radiation, one finds that besides the already examined 
application to: (i) the evaluation of the periods of the crystalline lattices, the X- 
radiation could be applied successfully to the: (ii) investigation of defects in various 
industrial components, (ii1) examination of different solid parts of the human body, 
(iv) medical treatment of certain diseases (as of some cancer affections), etc. 


Short Examples of other Important Purposes 


To achieve a brief presentation of other important purposes and of the basic 
contributors to their better knowledge and advancement, we will summarize the 
essential elements in the frame of the next table. 


N, P Main International Main Romanian Author’s first 
se ae aie Contributors Contributors Contributions 
Continuous X- 
1. Basie Pealures OF | | eB) Kaelbls 109] O. Birau [30] Raye Sa e 
rays Semantics [21], 
1960 
‘ Complex Magnetic 
oT Basic Features of K.G. Wilson [31] R. Dobrescu, D.I. Materials [33], 
Complex Materials [32] 
1967 
D. Barca-Galateanu, | NMR data proces- 
3. Molecular Structure G. Herzberg [34] M. Giurgea[35] sing [36], 1973 
4 Physics Structure L. A. Sena [37] 2 Ely sesnonons & 
y a methods [38],1975 
5 Information Accuracy — | Courant — Fr-Lewy = Pulse distortions 
: Numerical Phenomena [39] Delsanto [40] [41], 1997 
‘ R. Widenhorn, Various applica- 
OT, ll See CDS): Seiesede gam 142} oe tions [43], 2013 
4 Information about the R. G. LeTourneau = Annotated trans- 
life of US inventors [44] lation [45], 2016 
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